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DETAILED ACTION 

This action is in response to the communication filed on 1 1/03/2008. 

• Claims 1, 25, 28, 29, and 32 have been amended. 

• Claim 22 has been cancelled. 

Claims 1-21 and 23-32 are pending and have been examined. Previous objections to 
the claims have been withdrawn in view of amendments. 

Claim Rejections - 35 USC § 101 

1. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 

2. Claim 28 is rejected under 35 U.S.C. 101 because the claimed invention is 
directed to non-statutory subject matter. As per claim 28, the claim is directed to a 
system comprising "means for receiving", "means for determining and accessing", 
"means for enabling". It is noted that it appears that the term "means for" is merely 
placed before each of the method steps of claim 1 . One of ordinary skill would 
reasonably interpret each of such "means for" as merely software code, which does not 
fit any of the statutory classes of invention. 

Claim Rejections - 35 USC § 102 

3. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 1 02 that 
form the basis for the rejections under this section made in this Office action: 
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A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

4. Claims 1-21 and 23-32 are rejected under 35 U.S.C. 102(b) as being anticipated 
by Zimmerman (U.S. Patent Application Publication 2002/0193975). 

As per independent claim 1 , Zimmerman teaches a method for assisting 
navigation of digital content using a tangible medium, comprising: receiving an 
instruction to access digital content corresponding to a portion of a tangible 
medium (e.g. in figure 1 and in paragraph 20 on page 2, "The system generally 
comprises a probe device and a computer. The probe is 
manipulated over off-line media, preferably printed medium, and 
information is transmitted to the computer and matched to 
electronic media" and paragraph 26), said medium being readable by a user- 
positionable input device (e.g. in figure 1 and in paragraph 20 on page 2, "The 
system generally comprises a probe device and a computer. The 
probe is manipulated over off-line media, preferably printed 
medium, and information is transmitted to the computer and 

matched to electronic media"), and said digital content being accessible 
from a stored file (e.g. in paragraph 1 18 on page 7, " ah programming and data 

related thereto are stored in computer memory, static or 
dynamic, and may be retrieved by the user in any of: 
conventional computer storage (local or remote), display (i.e., 
CRT) and/or hard copy (i.e., printed) formats"); determining and 
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accessing stored digital content corresponding to said input device's 
instantaneous position on said tangible medium (e.g. in paragraph 21 on page 2, 

"Based upon the position of the probe over the off-line media, 
corresponding multimedia data is retrieved from storage and 
presented to the user "); and enabling electronic navigation of said digital 
content (e.g. in paragraph 21 on page 2, "Preferably, the multimedia data 
and off-line media are designed so as to provide an educational 
experience in which the synchronization of the probe position 
and output provides the visceral experience normally experienced 
by a user of a real tool. To simulate the real tool, the 
multimedia data corresponds to the output a real tool would 
produce when the real tool is used to inspect the items on the 
off-line media ", in paragraphs 77 and 90), including enabling toggling between 
browsing of said tangible medium and browsing of said digital content on a 
computer screen using said input device (e.g. in paragraphs 24, 77, 81, and 89-90, 
the user can move a finger, a probe or other device over the tangible medium onto a 
particular section, icon, word, etc. of the tangible medium, browsing the tangible 
medium; through this, digital content is retrieved which toggles to the browsing of the 
digital content (e.g. moving the frog's eye around on the computer screen, playing an 
audio or video clip); the user can then select a different section, icon, word, etc., 
toggling back to browsing of the tangible medium), said browsing of digital content 
being performed using said input device or a second input device, wherein said 
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browsing of digital content includes enabling a user to control translational or 
rotational movements as directed by the user (in paragraphs 70 and 81, and figure 
1)- 

As per claim 2, the rejection of claim 1 is incorporated and Zimmerman further 
teaches determining a change in position of said input device on said tangible 
medium; and obtaining a new stored file corresponding to said change in 
position (e.g. in paragraph 24 on page 2, "the printed medium additionally 
has a number of icons representing different tools to be 
simulated. By placing the probe over a particular icon , the tool 
represented by the icon is selected such that the retrieved 
multimedia data corresponds to the selected tool "). 

As per claim 3, the rejection of claim 1 is incorporated and Zimmerman further 
teaches wherein said determining and accessing stored digital content includes: 
obtaining digital signals representing a localized region of said tangible medium, 
said localized region being proximate to said position of said input device on said 
tangible medium (e.g. in paragraph 66 on page 4, "Referring to fig. 4, in a 

preferred embodiment of the present invention, probe 104 
contains a CMOS monochromatic camera 400 with wide angle lens 
406 and illumination system 410 to capture an image segment 408 
printed on the medium 106, underneath the probe . The image is 
detected and transmitted to the computer 100 by image processor 
4 02, communicating to the computer 100 through connector 404, 
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such as Universal Serial Bus (USB) "; the computer inherently obtains digital 
signals through a USB connector); determining at least one stored file 
corresponding to said localized region, and containing said digital content, by 
using pattern matching (e.g. in paragraph 66 on page 4, "The image segment 408 
is a portion of a full image 412 appearing in the off-line media 
106. An image retrieval method running in the computer 100 
receives the image segment 408, and outputs image identification 
and position information of the image segment 4 0 8" and in paragraph 68 
on page 4, "FIG . 5 shows a flow chart summarizing the methods of 
sampled image to reference image matching. . . At block 50 6 the 
salient features of the sampled image are extracted. At block 
508 the salient features of the sampled image are used to find 
the closest reference image " in view of figure 5; comparing salient features is 
one form of pattern matching); and retrieving an appropriate portion of said file to 
enable user navigation (e.g. in paragraph 21 on page 2, "Based upon the 

position of the probe over the off-line media, corresponding 
multimedia data is retrieved from storage and presented to the 
user... To simulate the real tool, the multimedia data corresponds 
to the output a real tool would produce when the real tool is 
used to inspect the items on the off-line media "). 

As per claim 4, the rejection of claim 3 is incorporated and Zimmerman further 
teaches wherein said pattern matching is based on correlating a pattern within 
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said localized region with a pattern in said stored file (e.g. in paragraph 68 on page 

4, "FIG. 5 shows a flow chart summarizing the methods of sampled 
image to reference image matching. . . At block 504 the sampled 
image is captured by the camera 400 in the probe 104. At block 
506 the salient features of the sampled image are extracted. At 
block 508 the salient features of the sampled image are used to 
find the closest reference image " in view of figure 5). 

As per claim 5, the rejection of claim 3 is incorporated and Zimmerman further 
teaches wherein said pattern matching is based on correlating a pattern 
embedded within said medium itself (e.g. in paragraph 20 on page 2, "The system 
generally comprises a probe device and a computer. The probe is 
manipulated over off-line media, preferably printed medium , and 
information is transmitted to the computer and matched to 
electronic media" and in paragraph 45 on page 3, "Printed, off-line 
medium 10 6 has a suitable image 108 imprinted thereon"). 

As per claim 6, the rejection of claim 3 is incorporated and Zimmerman further 
teaches wherein said tangible medium was previously created independently of 
said file (e.g. in paragraph 87 on page 6, "in an application of the 

invention where the media 106 includes text, the user may circle 
text, or swipe the text with the probe 104 to select, translate, 
provide definition (e.g. from a dictionary), underline, 
highlight, make bold, copy or cut the corresponding electronic 
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text appearing in the computer 100. In this application, images 
of the text, or salient features of the text need not be stored 
in the computer 100. In a preferred embodiment, text is stored 
as ASCII characters along with the font and other layout and 
contextual information to enable the computer to generate an 
accurate image of the page, or otherwise sufficiently represent 
the page"). 

As per claim 7, the rejection of claim 1 is incorporated and Zimmerman further 
teaches wherein said determining and accessing stored digital content includes: 
obtaining coordinates of said position of said input device on said tangible 
medium, determining at least one stored file corresponding to said position and 
containing said digital content, determining coordinates within said stored file, 
corresponding to said input device position coordinates, by using coordinate 
mapping ("The computer decodes the bar code information into probe 
position and image identification , resulting in a synchronized 
image 2 04 on the computer screen of human bones 2 0 6" in view of figures 
2a and 2b; identifying a position on an image inherently involves obtaining and 
determining coordinates, and in this example, the probe position (coordinate) on the 
tangible medium is directly mapped to a position on the stored image); and using said 
determined coordinates to retrieve an appropriate portion of said file to enable 

user navigation ("... resulting in a synchronized image 204 on the 

computer screen of human bones 2 0 6" in view of figure 2b). 
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As per claim 8, the rejection of claim 7 is incorporated and Zimmerman further 
teaches wherein said coordinate mapping involves a linear transformation from 
tangible medium coordinates to stored file coordinates (e.g. in paragraph 51 on 

page 3, "The computer decodes the bar code information into probe 
position and image identification, resulting in a synchronized 
image 2 04 on the computer screen of human bones 2 0 6" in view of figures 
2a and 2b; in this example, the probe position on the tangible medium is directly 
mapped to the position on a stored image, and therefore inherently involves a linear 
transformation from tangible medium coordinates to stored file coordinates). 

As per claim 9, the rejection of claim 7 is incorporated and Zimmerman further 
teaches wherein at least one of said tangible medium and said stored file includes 
a grid system (e.g. in paragraph 51 on page 3, "The computer decodes the bar 

code information into probe position and image identification, 
resulting in a synchronized image 204 on the computer screen of 
human bones 2 0 6" in view of figures 2a and 2b; identifying a position on an image 
inherently involves a coordinate or grid system). 

As per claim 10, the rejection of claim 7 is incorporated and Zimmerman further 
teaches wherein said determining said stored file includes utilizing a file index 
read from said tangible medium (e.g. in paragraph 49 on page 3, "fig. 2a 

illustrates the use of the present invention with position and 
image identification information encoded in bar codes printed on 
the off-line media "). 
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As per claim 1 1 , the rejection of claim 7 is incorporated and Zimmerman further 
teaches wherein a file index was previously generated during creation of said 
tangible medium (e.g. in paragraph 49 on page 3, "fig. 2a illustrates the 

use of the present invention with position and image 
identification information encoded in bar codes printed on the 
off-line media "). 

As per claim 12, the rejection of claim 7 is incorporated and Zimmerman further 
teaches wherein said tangible medium includes a plurality of machine-readable 
patterns embedded in said medium itself (e.g. in paragraph 45 on page 3, 
"Printed, off-line medium 106 has a suitable image 108 imprinted 
thereon" and in paragraph 49 on page 3, "FIG. 2a illustrates the use of 
the present invention with position and image identification 
information encoded in bar codes printed on the off-line media "); 

said obtaining coordinates of said position of said input device is based on 
reading a unique pattern at said position of said input device, and analyzing said 
unique pattern to determine said coordinates (e.g. in paragraph 51 on page 3, "The 

probe 104 is placed over the picture of the hand 200 in book. 
The probe 104 sends bar code information to the computer. The 
computer decodes the bar code information into probe position 
and image identification , resulting in a synchronized image 204 
on the computer screen of human bones 2 0 6" in view of figures 2a and 2b). 
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As per claim 13, the rejection of claim 1 is incorporated and Zimmerman further 
teaches wherein said digital content includes an image, and said navigation 
includes displaying said image (e.g. in paragraph 81 on page 5, "The invention 
matches the sampled image 70 6 to a reference image of the frog's 
eye retrieved from storage device 102. Based on the probe 104 
position, reference image, and tool selected, a magnified view 
of a frog eye 802 is presented on the computer screen 800. The 
invention synchronizes the movement of the probe 104 to the 
display of the frog's eye 802. When the user moves the tool to 
the right, the image of the frog's eye on the computer 800 
screen would also move to the right, giving the user the 
visceral experience of holding an actual microscope over a 
frog's eye" in view of figure 7). 

As per claim 14, the rejection of claim 1 is incorporated and Zimmerman further 
teaches wherein said digital content includes audio, and said navigation includes 
playing said audio (e.g. in paragraph 77 on page 5, "when the user 616 touches 

a particular word, for example "forest" 618 as show in FIG. 6, 
the computer 610 retrieving an audio clip of the spoken word 
"forest" from storage 614, and played it out speakers 620. When 
the user 616 touches the drawing 606, the sound of a bear 
growling occurs" in view of figure 6). 
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As per claim 15, the rejection of claim 1 is incorporated and Zimmerman further 
teaches wherein said tangible medium serves as a video storyboard (e.g. in 

paragraph 89 on page 6, "The remote server finds the best reference 
image match. In response to this match, the remote server sends 
a video clip 914 (for example MPEG1 compressed video) to the 
computer for presented on the computer display 912, in response 
to the sampled image in the magazine 90 4" in view of figure 9a and 9b). 

As per claim 16, the rejection of claim 1 is incorporated and Zimmerman further 
wherein said navigation includes at least one user-selectable mode (e.g. in 
paragraph 83 on page 5, "Since the simulated display on the computer 
screen is not bound by the static reality of off-line media, the 
synchronized presentation may include dynamic images. For 
example when viewing the eye, the user can press a button on the 
probe 104 (not shown) to generate a virtual bright light at the 
probe, causing the image of the frog's eye 802 to retract in the 
socket"). 

As per claim 17, the rejection of claim 16 is incorporated and Zimmerman further 
teaches wherein said modes are designated on, and selectable from, said tangible 
medium (e.g. in paragraph 24 on page 2, " the printed medium additionally 
has a number of icons representing different tools to be 
simulated. By placing the probe over a particular icon, the tool 
represented by the icon is selected such that the retrieved 
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multimedia data corresponds to the selected tool " and in paragraph 66 
on page 4, "A presentation method running in the computer 100 
receives the image segment 408 information and presents 
multimedia content contained in a CD-ROM storage device 102 
based on the virtual tool or function selected by the user " in view 
of figure 4). 

As per claim 18, the rejection of claim 1 is incorporated and Zimmerman further 
teaches wherein said tangible medium includes paper (e.g. in paragraph 21 on page 

2, "The probe is manipulated over off-line media, preferably 
printed medium , " and in paragraph 26 on page 2, "Off-line media can be 
any non-networked media or object including books, magazines, 
newspapers, posters, pictures , mosaics, tapestry, two and three 
dimensional objects, animals, people, furniture, toys, cups, 
plates, silverware, business cards , and clothing"). 

As per claim 19, the rejection of claim 1 is incorporated and Zimmerman further 
teaches wherein said input device includes an optical device (e.g. in paragraph 23 
on page 2, "The probe includes a camera to capture image segments 
and transmits them to a computing device"). 

As per claim 20, the rejection of claim 1 is incorporated and Zimmerman further 
teaches wherein said input device includes a radio frequency device (e.g. in 
paragraph 23 on page 2, " The probe includes a camera to capture image 
segments and transmits them to a computing device , either local 
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or remote, communicating through wired or wireless means " and in 
paragraph 45 on page 3, "... Probe device 104 is attached to computer 
100 via any suitable I/O interface so as to allow the probe to 
input data to computer 100. The I/O interface can including 
wired (e.g. USB, serial, parallel, firewire, optical 
communication) or wireless (e.g. 802.11, Bluetooth, UHF, 
infrared, CDMA, G3, PCS, mobile phone, ISM band RF) means "). 

As per claim 21 , the rejection of claim 1 is incorporated and Zimmerman further 
teaches wherein said tangible medium is two-dimensional, yet includes three- 
dimensional information (e.g. in paragraph 26 on page 2, "off-line media can 
be any non-networked media or object including books, magazines, 
newspapers, posters, pictures , mosaics, tapestry, two and three 
dimensional objects, animals, people, furniture, toys, cups, 
plates, silverware, business cards, and clothing. . ."). 

As per claim 23, the rejection of claim 1 is incorporated and Zimmerman further 
teaches wherein said tangible medium was previously created using said stored 
file (e.g. in paragraph 50 on page 3, "the bar codes 202 are printed on 
clear plastic (e.g. mylar) sheets and adhered to the pages of 
the off-line media. This allows a book to be published using 
conventional printing techniques, and bar codes to be placed on 

the pages after conventional printing "; the bar codes refer to the stored 
files likely used in the printing of the book). 
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As per claim 24, the rejection of claim 1 is incorporated and Zimmerman further 
teaches implemented in a handheld portable electronic device (e.g. in paragraph 

118, "the present invention may be implemented on a conventional 
IBM PC or equivalent, mobile phone , personal digital assistant 
(PDA) , tablet computer , multi-nodal system (e.g., LAN) or 
networking system (e.g., Internet, WWW)"). 

Claims 25, 26, and 27 are the computer-readable storage medium claims 
corresponding to the method claims 1 , 3 and 7 respectively, and are rejected under the 
same reasons set forth in connection with the rejection of claims 1 , 3, and 7. 
Zimmerman further teaches the computer-readable storage medium comprising 
logic instructions (e.g. in paragraph 118 on page 7, "The above system and its 
described functional elements are implemented in various 
computing environments... All programming and data related 
thereto are stored in computer memory , static or dynamic, and 
may be retrieved by the user..."). 

Claim 28 is the system claim corresponding to the method claim 1 and is rejected 
under the same reasons set forth in connection with the rejection of claim 1 . 
Zimmerman further teaches the means for receiving (e.g. in paragraph 45 on page 3, 
"Probe device 104 is attached to computer 100 via any suitable 
I/O interface so as to allow the probe to input data to computer 
100. The I/O interface can including wired (e.g. USB, serial, 
parallel, firewire, optical communication) or wireless (e.g. 
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802.11, Bluetooth, UHF, infrared, CDMA, G3, PCS, mobile phone, 
ISM band RF) means" and in paragraph 1 1 8 on page 7, "For example, the 
present invention may be implemented on... multi-nodal system 
(e.g., LAN) or networking system (e.g., Internet, WWW) "), means for 
determining and accessing (e.g. in paragraph 23 on page 2, "The computing 
device analyses the image segments, determines the image 
identity, and retrieves and presents electronic media 
corresponding to the image to the user"), and means for enabling (e.g. in 
paragraph 81 on page 5, "Based on the probe 104 position, reference 
image, and tool selected, a magnified view of a frog eye 802 is 
presented on the computer screen 800. The invention synchronizes 
the movement of the probe 104 to the display of the frog's eye 
8 02"). Therefore, Zimmerman teaches the system claimed. 

Claims 29, 30, and 31 are the system claims corresponding to the method claims 
1 , 3 and 7 respectively, and are rejected under the same reasons set forth in connection 
with the rejection of claims 1 , 3, and 7. Zimmerman further teaches the interface (e.g. 
in paragraph 45 on page 3, "Probe device 104 is attached to computer 
100 via any suitable I/O interface so as to allow the probe to 
input data to computer 100. The I/O interface can including 
wired (e.g. USB, serial, parallel, firewire, optical 
communication) or wireless (e.g. 802.11, Bluetooth, UHF, 
infrared, CDMA, G3, PCS, mobile phone, ISM band RF) means") and 
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processor (e.g. in paragraph 23 on page 2, "The computing device analyses 
the image segments, determines the image identity, and retrieves 
and presents electronic media corresponding to the image to the 
user"; a computing device inherently comprises a processor). Therefore, Zimmerman 
teaches the system claimed. 

As per independent claim 32, Zimmerman teaches a method for assisting 
navigation of digital content using a tangible medium, comprising: receiving an 
instruction to access digital content corresponding to a portion of a tangible 
medium (e.g. in figure 1 and in paragraph 20 on page 2, "The system generally 
comprises a probe device and a computer. The probe is 
manipulated over off-line media, preferably printed medium, and 
information is transmitted to the computer and matched to 
electronic media" and paragraph 26), said medium being readable by a user- 
positionable input device (e.g. in figure 1 and in paragraph 20 on page 2, "The 
system generally comprises a probe device and a computer. The 
probe is manipulated over off-line media, preferably printed 
medium, and information is transmitted to the computer and 

matched to electronic media"), and said digital content being accessible 
from a stored file (e.g. in paragraph 118 on page 7, " ah programming and data 

related thereto are stored in computer memory, static or 
dynamic, and may be retrieved by the user in any of: 
conventional computer storage (local or remote), display (i.e., 
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CRT) and/or hard copy (i.e., printed) formats"); determining and 
accessing stored digital content corresponding to said input device's 
instantaneous position on said tangible medium (e.g. in paragraph 21 on page 2, 

"Based upon the position of the probe over the off-line media, 
corresponding multimedia data is retrieved from storage and 
presented to the user"); enabling electronic navigation of said digital 
content, said electronic navigation of said digital content being performed using 
said input device or a second input device (e.g. in paragraph 21 on page 2, 
"Preferably, the multimedia data and off-line media are designed 
so as to provide an educational experience in which the 
synchronization of the probe position and output provides the 
visceral experience normally experienced by a user of a real 
tool. To simulate the real tool, the multimedia data corresponds 
to the output a real tool would produce when the real tool is 
used to inspect the items on the off-line media ", in paragraphs 77, and 
90), and enabling use of multiple tangible media to facilitate three-dimensional 
navigation (e.g. in paragraph 72 on page 5, "when three sequential sampled 
images are identified as belonging to the same reference image, 
a match is declared", in paragraph 22 on page 7, "the multimedia data and 
off-line media is designed so as to provide a commerce 
experience in which the synchronization of the off-line media 
and electronic material enhances the knowledge of the user to 
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the products in the off-line media, for example, showing 
different views or colors of a dress, or a simulation of the 
user in the dress ", in paragraphs 26, 63, 81 and 82, and figure 1), wherein said 
browsing of digital content includes enabling a user to control translational or 
rotational movements as directed by the user (in paragraphs 70 and 81, and figure 
1)- 



Response to Arguments 

5. Applicant's arguments filed 1 1/03/2008 have been fully considered but they are 
not persuasive. 

While it was agreed that paragraph 27 does not teach translation or rotation of 
the digital content, only translation or rotation of the probe, after further review of the 
cited prior art, Zimmerman clearly teaches controlling translation or rotation of the digital 
content in paragraphs 70 and 81. 

It is noted that the language of the claims suggests or makes optional but does 
not require a second input device and, as such, does not limit the scope of the claim. 

It is further noted that the use of the term "enabling" in the claims suggests or 
makes optional but does not appear to require the navigational steps and, as such, 
would not limit the scope of the claim. 



Conclusion 
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6. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 



US 6006126 A 
US 6164541 A 
US 6222583 B1 

US 20020158921 A1 
US 20020174105 A1 

US 20020173906 A1 
US 6664956 B1 



System and method for stereotactic 
registration of image scan data 



Methods and systems for providing 
human/computer interfaces 



Cosman; Eric R. 



Dougherty; Thomas J. 
etal. 



Matsumura; Takahiro 
Device and system for labeling sight images et al. 

Method and apparatus for virtual oversized 
display using a small panel display as a 
movable user interface 



Silverstein, D. Amnon 



Method for storing records at easily 
accessible addresses 

Portable navigation device and system, and 
online navigation service in wireless 
communication network 

Method for generating a personalized 3-D 
face model 



De La Huerga, Carlos 



Muramatsu, Toshihiko 



Erdem; A. Tanju 



7. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to WILLIAM WONG whose telephone number is 571-270- 
1399. The examiner can normally be reached on M-F 8:30-5:00 PM EST. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Stephen Hong can be reached on 571-272-4124. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/William Wong/ 
Examiner, Art Unit 2178 



/Adam L Basehoar/ 

Primary Examiner, Art Unit 2178 



